Echos from the Black Box

Counterfactual Explanations and Predictive Uncertainty for Trustworthy Machine Learning

Patrick Altmeyer

Delft University of Technology

Arie van Deursen
Cynthia C. S. Liem

May 18, 2023

Quick Introduction

  • Currently 2nd year of PhD in Trustworthy Artificial Intelligence at Delft University of Technology.
  • Working on Counterfactual Explanations and Probabilistic Machine Learning with applications in Finance.
  • Previously, educational background in Economics and Finance and two years at the Bank of England.
  • Enthusiastic about free open-source software, in particular Julia and Quarto.

Background

Counterfactual Explanations

Counterfactual Explanation (CE) explain how inputs into a model need to change for it to produce different outputs.

Provided the changes are realistic and actionable, they can be used for Algorithmic Recourse (AR) to help individuals who face adverse outcomes.

Example: Consumer Credit

In Figure 1, arrows indicate changes from factuals (loan denied) to counterfactuals (loan supplied).

Figure 1: Counterfactuals for Give Me Some Credit dataset (Kaggle 2011).

Predictive Uncertainty

Predictive Uncertainty (PU) is a measure of how uncertain a model is about its predictions.

We have been exploring both Bayesian (Laplace Redux) and Frequentist approaches (Conformal Prediction) in our work.

Example: Laplace Redux

LaplaceRedux.jl is our Julia package for Bayesian Neural Networks (BNNs) with Laplace Approximation.

Figure 2: Laplace Redux for a simple Multi-Layer Perceptron.

Example: Conformal Prediction

ConformalPrediction.jl is our Julia package for Conformal Prediction.

Figure 3: Conformal Prediction sets for an Image Classifier.

ECCCos from the Black Box

Pick your Poison?

All of these counterfactuals are valid explanations for the model’s prediction. Which one would you pick?

Figure 4: Turning a 9 into a 7: Counterfactual Examplanations for an Image Classifier.

What do Models Learn?

These images are sampled from the posterior distribution learned by the model. Looks different, no?

Figure 5: Conditional Generated Images from the Image Classifier

ECCCos

We propose a framework for generating Energy-Constrained Conformal Counterfactuals (ECCCos) which explain black-box models faithfully.

Figure 6: (a) Wachter, (b) ECCCo (no EBM), (c) ECCCo (no CP), (d) ECCCo

Figure 7: Gradient fields and counterfactual paths for different generators.

Trustworthy AI in Julia

🐶 Taija

Research informs development, development informs research.

Trustworthy Artificial Intelligence in Julia.

Taija is a collection of open-source packages for Trustworthy AI in Julia. Our goal is to help researchers and practitioners assess the trustworthiness of predictive models.

Our work has been presented at JuliaCon 2022 and will be presented again at JuliaCon 2023 and hopefully beyond.

References

Kaggle. 2011. “Give Me Some Credit, Improve on the State of the Art in Credit Scoring by Predicting the Probability That Somebody Will Experience Financial Distress in the Next Two Years.” Kaggle. https://www.kaggle.com/c/GiveMeSomeCredit.